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Abstract 

We study causal dynamic approximation of non-bandlimited processes by band-limited 
processes such that a part of the historical path of the underlying process is approximated 
in L2-norm by the trace of a band-limited process. This allows to cover the case of irregular 
non-smooth processes. We show that this problem has an unique optimal solution. The 
approximating band-limited process has unique extrapolation on future times and can be 
interpreted as a optimal forecast. To accommodate the current flow of observations, the 
selection of this band-limited process has to be changed dynamically. This can be interpreted 
as a causal and linear filter that is not time invariant. 
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1 Introduction 

We study causal dynamic approximation of non-bandlimited processes by band-limited pro- 
cesses. It is known that it is not possible to find an ideal low-pass causal linear time-invariant 
filter. It is also known that the distance of the set of these ideal low-pass time invariant filters 
from the set of all causal filters is positive [I]. In addition, it is known that optimal approx- 
imation of the ideal low-pass filter is not feasible in the class of causal linear time-invariant 
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filters (see, e.g., [3] and references here). In the present paper, we are trying to substitute the 
solution of these unsolvable problems by solution of an easier problem where the filter is not 
necessary time invariant. Our motivation is that, for some problems, time invariancy for a filter 
is not crucial. For example, a typical approach to forecasting in finance is to approximate the 
known path of the stock price process by a smooth process that has an unique extrapolation 
and accept this extrapolation as the forecast. This procedure has to be done at current time; 
it is nor required that the same forecasting rule will be applied at future times. We apply this 
approach with the band-limited processes used as approximating smooth predictable processes. 
More precisely, we suggest to approximate in L2-norm the known historical path of the process 
by the trace of a band-limited process. In this setting, the approximating curve does not nec- 
essary match the underlying process at given sampling points. This is different from classical 
sampling approach (see, e.g., [7]). Similarly to [l]-[5], our setting allows to cover the case of 
irregular non-differentiable or discontinuous processes such as historical stock prices in continu- 
ous time models. The difference is that [1]-[S] achieves point- wise matching for the underlying 
process being smoothed by a convolution operator; we consider approximation of the underlying 
process directly using different methods. In [4j-[5j, the estimate of the error norm is given. In 
our setting, it is guaranteed that the approximation generates the error of the minimal norm. 

We show that an unique optimal solution of approximation problem exits. The approximat- 
ing process is derived in time domain in a form of sine series. To accommodate the current 
flow of observations, the coefficients of these series and the related band- limited processes have 
to be changed dynamically. It can be interpreted as a causal and linear filter that is not time 
invariant. 

2 Definitions 

We denote by L,2(D) the usual Hilbert space of complex valued square integrable functions 
x : D — > C, where D is a domain. 

For x(-) £ L2(R), we denote by X = Tx the function defined on iR as the Fourier transform 
of x(-); 

/oo 
e~ iujt x(t)dt, weR. 
-oo 

Here i = y— 1. For x(-) 6 L2(R), the Fourier transform X is defined as an element of ^2(R) 
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(more precisely, X(i-) G L 2 (R)). 

For a given Cl > 0, let Uq iOQ = {X(ica) G L 2 (iR) : X(ioj) = for |cj| > Cl}, and let Uq^ 
be the set of all X G J7q )0 o such that there exists a sequence {yfe}fcL_Ar £ (^2iv+i suc ] 1 ^at 
= X^fcL-TV yke lku> ^ n ^{\uj\<n}i where I is the indicator function. 

For N = +00 and for integers N > 0, consider Hilbert spaces y N such that y N = C 2N+1 for 
N < +00 and 3V is the set of all sequences {yk}k=-N e C 2N+1 such that l Cfe l 2 < 

Let s G R and g < s be given; the case when q = —00 is not excluded. Consider Hilbert 
spaces of complex valued functions X = L 2 (— 00, +00) and X_ = L 2 (q, s). 

Let > and iV be given (the case of N = +00 is not excluded). Let Xq jy be the subset 
of <Y_ consisting of functions sc|(g )S ], where x £ X are such that ir(t) = (J-~ 1 X)(t) for i G [q,s] 
for some X(iu) G Wn,AT. 

Proposition 2.1 For any cc G ^njv? ^ere exists an unique X G Wn jv swc/z f/iaf x(t) = 
(T^XXt) forte [q,s). 

For a Hilbert space H, we denote by (•, -)h the corresponding inner product. We use notation 
sine (x) = sin(x)/x. 

3 Main results 

3.1 Optimal band- limited approximation 

Let x £ X be a process. We assume that the path x(s)| sg [ 9)S ] represents available historical data. 
Let Hermitian form F : Xq^ x — > R be defined as 

F(x,x) = [ \x(t) - x(t)\ 2 dt. 

Jq 

Theorem 3.1 For any N < +00, there exists an unique solution x of the minimization problem 

Minimize F(x,x) over x G Xq^- (3-1) 

Remark 3.1 By Proposition \2.1\ there exists an unique extrapolation of the band-limited so- 
lution x(t) of problem jl3. 1\) on the future time interval (s, +00). It can be interpreted as the 
optimal forecast (optimal given Cl and N). 
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3.2 Optimal sine coefficients 

To solve problem (|3.ip numerically, it is convenient to expand X{ioj) via Fourier series. 

For a given Q > 0, consider the mapping Q : —> Xq,n such that x = Qy is such that 
x(t) = (J r_1 X)(t) for a.e. t S (q, s], where 

N 

X(iu) = yte ltui/ %u\<n } . 

t=-N 

Clearly, this mapping is linear and continuous. 

Let Hermitian form G : x X- — >• R. be defined as 

G{y,x) = F{Qy,x) = [ \x(t) - x(t)\ 2 dt, x = Qy. (3.2) 

J a 



Corollary 3.1 For any N < +oo ; there exists an unique solution y of the minimization problem 

Minimize G(y,x) over y G 3^v- (3-3) 



Problem (|3.ip can be solved via problem f|3.3j) : its solution with N < +oo can be found 
numerically. 

3.3 Solution of problem ( ET5j) 

Let iV be given, let Z be the set of all integers z such that \z\ < N if N < +oo, and let Z be 
the set of all integers if N = +oo. Let 

kez 

where G 3^jv- Let a? = -T 7-1 ^ • We have that 

\k£Z / k^Z 
gi/c7rH-2r2t ^—ikir—iflt q 

= 2^ E yfc — »wn + it — = 7 £ yfcSinc (fc7r + nty 

kez 1 kez 
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Remark 3.2 Let t[k] = —kir/ft. Clearly, x = J- X is such that x(i[fc]) = y k ■ Q/ir, i.e., 
Uk = x(t[k]) ■ ir/Q, and, therefore, 

x(t) = ^2 x(t[k])smc (kir + tit). 
It gives celebrated Sampling Theorem; see, e.g., [7]. 

Remark 3.3 We consider a setting when only the part x(t)\ t& t q ^ of the path of the process 
is available at current time s < +oo. In this setting, sampling theorem is not applicable. Our 
approximation can be considered as a modification of the truncated sine approximation (see, 
e.g., [6], [7J). The difference is that the increasing of N is not related to extension the time 
interval [q, s] in our setting. 

We have that 

n 2 

— } 2/fcsinc (kir + Ut) — x(t) 
7T ^— ' 

k&Z 



G(y,x) = / \x(t) - x(t)\ 2 dt = 

J q 



dt 

(y, Ry)y N ~ 2Re (y> rx )x_ + {px, x) x _ . (3.4) 



Here R : y^ x J^v — > is a linear bounded Hermitian operator, r : AL_ — > 3V is a bounded 
linear operator, p : X- x — > X- is a linear bounded Hermitian operator. 

It follows from the definitions that the operator R is non-negatively defined (it suffices to 
substitute x(t) = into the Hermitian form). 

3.4 The case when iV < +00 

Up to the end of this paper, we assume that N < +00. In this case, the space y^ is finite 
dimensional, it follows that the operator R can be represented via a matrix R = {Rkm} £ 
C 2N+1 ' 2N+1 , where R km = R mk and (Ry) k = J2k=-N R k m y m - 

Theorem 3.2 (i) For any N < +00 , the operator R is positively defined, 
(ii) Problem h3. 3\) has a unique solution y = R~ l rx. 
( Hi ) The components of the matrix R can be found from the equality 

Rkm = / sine (mir + r^)sinc (kir + Qt)dt. (3-5) 

Jq 
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(iv) The components of the vector rx = {(rx)f.}i 



■=-N 



can be found from the equality 



(rx)k = — / sine (kir + Qt)x(t)dt. 

7T Jq 




(3.6) 



Corollary 3.2 Let y = y(q,s) be the vector calculated as in 

TheoremlEM V = {m}k=-N- The 

process 



x(t) = x(t, q, s) = — y yfcsinc (kir + Qt) 

kez 

represents the output of a causal filter that is linear but not time invariant. 

4 Numerical experiments 

In the numerical experiments described below, we have used MATLAB symbolic integration 
for calculation of integrals fj3.5j) and (|3.6|) . The experiments show that some eigenvalues of R 
are quite close to zero. Because of the integration errors, some eigenvalues of the calculated 
matrix R are actually fluctuating around zero despite the fact that, by Theorem 13.21 R > 0. 
Respectively, the error E = \\Ry — rx\\i 2 ^ q ^ \ for the MATLAB solution of the equation Ry = rx 
does not vanish. This error depends on the error tolerance parameter tol of MATLAB integration 
operator QUAD that was used; the default value is tol = 1CP 6 ; we used tol = 1CP 8 . Further, in 
our experiments, we found that the error E can be decreased by the replacing R in the equation 
x = R~ 1 rx by R e = R + el, where / is the unit matrix and where e > is small. In particular, 
for e = 0.001, the corresponding error E{e) = \\Rj l rx — y||wg )S ) < \\R^rx — y\\L 2 (q,s)i i- e -> the 
approximation on [q, s] is better for y = R~ l rx calculated for e = 0.001 than for y = R~ 1 rx 
calculated for s = 0. 

Figures [5.11 and [5T21 show examples of a process x(t) and the band-limited process x(t) approx- 
imating x(t) on time intervals (q, s] = (—12,-2] and (q, s] = (—10,0], respectively, calculated 
with e = 0.001 for SI = 4 and ./V = 30. As expected, the change of the time interval from 
(q, s] = (—12,-2] to (q, s] = (—10,0] results in the change of the approximating band-limited 
process. 

Note that the experiments demonstrate robustness with respect to the changes of N. The 
curves of x(t) will be almost the same if we consider = 50 instead of N = 30, when all other 
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parameters are the same. However, the error E is larger for large N = 100, due to accumulated 
larger error of integration. 

The shape of curves of x(t) depends on the choice fi. Figure [573] shows an example of a process 
x(t) and of the band-limited process x(t) approximating x(t) on time interval (q, s] = (—10,0] 
calculated for 17 = 2, when all other parameters are the same as for Figure [5721 

By Remark l3.1[ the extrapolation of the process x S Xq.,n on the future time interval (s, +oo) 
can be interpreted as the optimal forecast (optimal given 0, and N). 

Remark 4.1 We have used the procedure of replacement R by R £ = R + el with small e > to 
reduce the error of calculation of the inverse matrix for the matrix R that is positively defined 
but is close to a degenerate matrix. It can be noted that the same replacement could lead to a 
meaningful setting for the case when e > is not small. More precisely, it leads to optimization 
problem 

N 

Minimize G(y,x)+e 2 \yk\ 2 over y G 3V- (4-1) 

k=-N 

The solution restrains the norm of y, and, respectively, the norm of x. 

Figure [5741 illustrates Remark 14.11 with an example of a process x{t) and the corresponding 
band-limited process x{i) calculated via solution of problem (14. lh for e = 0.05, when all other 
parameters are the same as for Figure 15.21 This solution was obtained by replacement of R by 
R £ = R + eI with e = 0.05. 



5 Appendix: proofs 



Proof of Proposition \2.1\ The statement of this proposition is known in principle. It suffices to 
prove that if x(-) £ Xn,N is such that x(t) = for t £ (q,s], then x(t) = 0. For the sake of 
completeness, we give below a proof. For C > 0, consider a class M(C) of infinitely differentiable 
functions x(t) : R — > R such that there exists M = M{x{-)) > such that 

<C k M, k = 0,1,2,.... 



d k x 



L 2 (R) 



Let Ai = Uc>oA / l(C). Any x £ M is infinitely differentiable and such that there exists C\ 
Ci(x(-)) > and M 1 = M 1 {x{-)) > such that 



sup 

t 



d k x , . 



Clearly, n C Ai. Therefore, any x(-) G <^h jv is analytic and allows the Taylor series 
expansion at any point with an arbitrarily large radius of convergence. Consider the Taylor 
series expansion at to G (q, s). Since all derivatives at this point are equal to zero, the expansion 
is identically equal to zero. This completes the proof of Proposition 12.11 □ 

Proof of Theorem \3.1\ It suffices to prove that Xq„n is a closed linear subspace of L2(q,s). 
In this case, there exists a unique projection x of a;|u a i on Xn,N, an d the theorem is proven. 

Clearly, for any N < +00, the set Uu,n is a closed linear subspace of Z^CR)- Consider a 
mapping Q : Uq^n — > Xq,n such that x(t) = (QX)(t) = (J- X)(t) for t G [q,s]. It is a linear 
continuous operator. By Proposition 12.11 it is a bijection. Since this mapping is continuous, 
it follows that the inverse mapping Q~ l : Xq n — > f7n,7V is also continuous (see Corollary in 
Ch.II.5 [8], p. 77). Since the set E/n,iV is a closed linear subspace of L2(R), it follows that Xq„n 
is a closed linear subspace of X- . This completes the proof of Theorem 13.11 □ 

Proof of Theorem \3.S\ Let us prove statement (i). We know that R > 0. Suppose that there 
exists y £ Q 2N + 1 such that y / and Ry = 0. Let r* : — > X_ be the adjoint operator to the 
operator r* : X_ — > 3^v- If r*y 7^ then there exists x G X_ such that G(y, x) < 0, which is not 
possible since G(y,x) > for all y, x. Therefore, r*y = 0, i.e., G(y,x) = {px,x)x_. Further, let 
y be a solution of problem (|3.3p . We have that G(y,x) = G(y + y,x). Hence y + y 7^ y is another 
solution of problem (13. 3|) . This contradicts to Corollary 13.11 that states that this problem has 
an unique solution. Statement (ii) follows from (i) and from classical theory of quadratic forms. 
Statements (iii)-(iv) follow immediately from representation (|3.4p . This completes the proof of 
Theorem □ 
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Figure 5.1: Example of x(t) and band-limited process x(t) approximating x(t) on (q, s] = (—12, —2], 
with Q = 4, and N = 30. 
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Figure 5.2: Example of x(t) and band-limited process x(t) approximating x(t) on (q, s] = (—10,0], 
with Q = 4, and N = 30. 




Figure 5.3: Example of x(t) and band-limited process x(t) approximating x(t) on (q, s] = (—10,0], 
with Q = 2, and N = 30. 
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